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0\ (54) Title: AN IMPROVED AUDIO AND DATA COLLABORATION AND COORDINATION SYSTEM 

^ (57) Abstract: There is disclosed a system and method for both initiating and perpetuating IP or non-IP audio calls with data collab- 
oration sessions over a network, such as the Internet (24). This is accomplished by providing an identifying field marking an audio 

Q call between caller to intended recipient. This field is then stored as an identifying marker in a data collaboration server (44), identi- 
fying a conversation which only ends when the audio call ends. Thus, the existence of the data collaboration session is not dependent 

^ on actual connection of the recipients to the data collaboration server (44) rather it exists so long as the audio link is maintained. 
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AN IMPROVED AUDIO AND DATA COLLABORATION AND COORDINATION 

SYSTEM 



CROSS REFERENCES TO RELATED APPLICATIONS 

This patent application is related to commonly owned U.S Provisional 
Patent Application No. 60/124,369, filed on March 15, 1999, entitled: Enabled Call 
Center System and to commonly owned PCT Patent Application stemming 
therefrom entitled "Audio and Data Collaboration and Coordination System " and 
filed on even date herewith, both of these applications are incorporated by 
reference herein. 

FIELD OF THE INVENTION 

The present invention is related to voice and data applications over 
communications networks, particularly over Internet Protocol (IP) networks and 
more particularly to systems and methods where a data collaboration session is 
maintained. 

BACKGROUND OF THE INVENTION 

The Internet is fast becoming an established way to do business, 
particularly in the field of the sale of goods and services, in what are termed 
"e-commerce" transactions. Typically, a business will have a web site which 
advertises the products by way of illustrations coupled with a method and mode of 
payment over the Internet. These modes of conducting business may be 
supplemented or complemented by traditional methods where a customer calls 
an operator at a company's call-center to order goods and services. When the 
two methods are combined a joint telephony/cosurfing (or data collaboration) 
conversation is utilized between a caller and an operator in the call center 
enabling the operator to show the customer around the web-site, whilst 
conversing with the caller. 

Co-surfing or data-collaboration may be defined as both parties seeing ~ 
the same pages loaded from the Internet as either party navigates through the 
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Internet and/or seeing the same data in Internet-based forms when either party 
enters data in a form. This is the case when an operator assists a customer in 
filling out information in a form on screen. Thus, the same pair of people talking in 
a telephony conversation should be connected in a cosurfing "conversation". 

5 Traditionally, the above has been achieved by an audio call being made 

between a client and the agent over the Internet. The audio call typically travels 
through a Public Switched Telephony Network to Internet Protocol or PSTN-to-IP 
gateway and then to the PSTN PBX or telephony switch, although it may travel to 
the PBX via conventional telephony means. The Client and Agent then speak 

10 and agree to co-surf ("agreement" may or may not be automatic) whereby the 
Client and Agent connect to the Internet by both connecting to a co-surfing server. 
Co-surfing data, that is the data that is to be viewed by both parties, then streams 
through a co-surfing server. Traditional Internet protocols are utilized to identify 
both parties in the co-surfing conversation. 

15 The co-surfing link is, however, inherently unreliable, given the limitations 

of a packet-switched network such as IP the possibility that one cosurfer might 
not have an open browser when the audio call starts and the possibilities that one 
cosurfer might close his browser during the call or indeed that the cosurfer's 
computer might crash during the audio call. The cosurfing conversation is thus 

20 susceptible to unreliability or to a premature end, premature being defined as 
ending before the audio call and the end of the transaction. 
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SUMMARY OF THE INVENTION 

The present invention improves on the contemporary art by providing 
systems and methods for both initiating and perpetuating IP or non-IP audio calls 
with data collaboration sessions over a network, such as the Internet. This is 
accomplished by providing an identifying field marking an audio call between 
caller to intended recipient. This field is then stored as an identifying marker in a 
data collaboration server, identifying a conversation which only ends when the 
audio call ends. Thus, the existence of the data collaboration session is not 
dependent on actual connection of the recipients to the data collaboration server 
rather it exists so long as the audio link is maintained. 

Embodiments of the invention are directed to a method of communication 
between at least two entities having at least two types of communications 
system, which an at least one communication system employs a first type of 
communication by a first communication application and a second 
communication system employs a second type of communication by a second 
communication application. The method includes the steps of, initiating 
communications between the at least two entities characterized by: establishing 
communications of a first type between the at least two first communication 
applications of the at least two of said entities; and sending a message for 
notifying of a start of communications of a first type between the at least two of 
the entities to a server of the communication system of second type for enabling 
communication of the second type. Communications between the at least two 
entities are terminated by, sending a message for notifying of an end 
communications of the first type between the at least two entities to the server 
of the communication system of the second type for disabling the second type 
of communication. 

Embodiments of the present invention also include a communication 
system for providing communications between an at least two entities, which the 
communications include an at least two types of communications and wherein 
the system comprises, first means for providing a first type of communications 
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and second means for providing a second type of communications. There is 
also an entity from the at least two entities for initiating the communications, 
wherein the first means is configured for; providing a first communication 
application for establishing communications of a first type; and providing a 

5 communication server for sending a message of a start of a first type of 
communications between at least two of the entities to a server of the second 
type of communication system for enabling the second type of communication 
and for sending a message of an end of the first type of communications to the 
server of the second communication system for disabling the second type of 

10 communication. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

The present invention will be understood and appreciated more fully from 
the following detailed description taken in conjunction with the appended drawings 
in which: 

Fig. 1 is a schematic illustration of an embodiment of the present 
invention in use in an exemplary application; 

Fig. 2 is a flow chart of a process useful in implementing the embodiment 
of the present invention; 

Figs. 3A-3B are UML object diagrams illustrating the information handled 
by the event handler of the DC server illustrated in Fig.1 hereinabove. 

Fig. 4 is a schematic illustration of a second embodiment of the present 
invention in use in an exemplary application; 
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DETAILED DESCRIPTION OF THE PRESENT INVENTION 

Reference is now made to Fig. 1 which is an illustration of a first 
embodiment of the present invention in use in a call center which is configured to 
allow a client 20 to call in and be served by an agent or operator 22a-22c. Clients 
and agents may collectively be termed entities. The client and agent sides are 
connected via a wide area network (WAN), typically the Internet 24. 

The Client 20, has a multimedia PC 30 (e.g Pentium ® ) with voice and 
data capabilities. There may also be more than one client, for example, a second 
Client 21 with a multimedia PC 33 as also illustrated on Fig.1. The multimedia PC 
30 employs an operating system such as Windows® NT® or the like, and is 
equipped with a suitable modem for accessing a wide area network (WAN), here 
the Internet 24. The PC 30, with monitor 31, is also loaded with software that 
operates as a browser for the internet , exemplary browsers suitable for use here 
include Microsoft® Internet Explorer® .Netscape® Navigator ® and Netscape 
Communicator ®. PC 33 is similarly configured to PC 31 . 

On the agent side, are web-enabled agents 22a-22c. While three agents 
are shown, this is exemplary only, for any number of agents (one or greater) is 
permissible in accordance with the present invention. These agents 22a-22c are 
typically equipped with multimedia PC's 32 with voice and data capabilities and 
include browsers, in accordance with those detailed above. The PC's of the 
agents 22a-22c have a connection to the WAN, here the Internet 24 for data, and 
connect to a Private Branch Exchange System (PBX) 36 for voice. The PBX 
system typically connects to the Internet 24, typically through a gatekeeper 38, for 
example a Vocal Tec Gatekeeper® from VocalTec Communications, Ltd Herzlia 
46733, although a gatekeeper is not necessary. The PBX system does not 
necessarily connect to the Internet, the vocal connection can be entirely separate 
from the Internet as described hereinbelow in relation to the second embodiment. 
Within the PBX 36 is an automatic call distribution unit (ACD) 40, that routs voice 
calls to the selected agent 22a-22c, typically the first available agent. For 
example, one ACD suitable for use with this system is a Definity G3 ACD from 
Lucent Technologies, Murray Hill, New Jersey 
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The PBX connects to a T-server 42. The T-server 42 connects to a Data 
Collaboration (DC) server 44 containing an event handler 45. DC server 44 also 
contains a message distributor, 47, for switching messages between co-surfers. 
The DC server 44 in turn connects to the Internet 24. 

Event handler 45 records and analyzes "events" such as the audio 
connection of a Client 30 and an Agent 22. This event (hereinafter "startcall event 
") indicates a potential to connect in a data collaboration connection (hereinafter 
"data collaboration" or "co-surfing") between the Client 30 and the agent 22 when 
the Client's 30 and agent's 22 browsers are connected. The potential to connect 
is hereinafter termed "a conversation" as it exists so long as the audio connection 
is maintained. The initiation of conversations and related data collaborations 
which is the subject of this patent will be described further hereinbelow. 

The DC server 44 typically includes a Computer Telephone Integration 
(CTI) driver 46, and can be a VocalTec Surf & Call ™ Server (available from 
VocalTec Communications Ltd, Herzelia 46733, Israel), that provides the interface 
between an incoming client (customer) call, telephony gateway terminals, call 
center equipment such as the ACD/switch and the internet 24. CTI driver 46 
typically contains the event handler 45. The T-server may be an Internet 
Telephony Server from Lucent Technologies, Murray Hill, New Jersey and serves 
to convert PBX telephony data into Internet Protocol (IP) data. 

Servers 1 and 2, indicated as 50,52 are connected to the Internet 24. 
These servers 50,52 are exemplary of all the servers connected to the Internet. 

One server, for example Server 1, 50 hosts a web-site that includes a 
downloadable software that allows for the contemporaneous surfing of the 
Internet together with the placement of an audio call by Client to the agent from 
an accessed web page. One exemplary software package suitable for this 
application is Surf & Call Center ™, having the Surf & Call ™ software, an 
embedded plug-in enabling web-to-phone call center applications from a standard 
web page, both products available from VocalTec Communications, Herzelia, 
Israel. 

Another server, for example, server 2, 52 may serve as a random number 
generator, discussed in co-pending patent application entitled "Audio Data 
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Collaboration and Coordination System", mentioned hereinabove and filed on 
even date. The random number generation is an exemplary method of 
co-ordinating the data collaboration session with the audio session but other 
methods as particularly illustrated hereinbelow in relation to the second 
embodiment may equally apply. 

Reference is now made to Fig. 2 which is a flow diagram to assist in 
describing a possible implementation of the process of the present invention. It 
should be noted that steps after step 70 can occur in any order. Reference is also 
further made to Fig. 1. The user or client 20, his browser open, has downloaded, 
or obtained by other means, software for the placement of voice calls between 
client and agent directly from agent's web-site. Here, for example, the client 20 
has used the Surf & Call Center ™ , having the Surf & Call ™ software, an 
embedded plug-in, as detailed above. This downloaded software appears on the 
client's monitor 31 as a rightmost portion which is a calling or audio call 
component 60. The browser of the Client is open and utilized to find the web 
page of the call center and typically an applet of the browser contains a data 
collaboration (DC) component 61, shown as the leftmost component on the 
Client's monitor 31, for the data call. 

A randomly, or otherwise generated ANI (Automatic Number 
Identification) is used to identify the customer to the telephony system and to the 
Data Collaboration System (cosurfing system). ANI s may be generated as serial 
numbers or taken from a Database. The generation of the ANI is described 
more fully in co-pending PCT patent application entitled " Audio and Data 
Collaboration and Coordination System" described hereinabove and filed on even 
date (hereinafter the "Collaboration Patent") and in commonly owned U.S 
Provisional Patent Application S/N 60/124,369, filed on March 15. 1999, entitled: 
Enabled Call Center System (hereinafter the "Call Center Patent "), the 
Collaboration Patent and the Call Center Patent incorporated by reference herein. 
It is to be emphasized herein that the method of identification is merely 
exemplary, other methods being available, the advantage of the ANI being that it 
replaces traditional telephony and data identifiers with one identifier which may be 
used for the Data Collaboration and Audio calls of the Client. 
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The agent, on the other hand, may be identified by a telephone 
extension. The ANI is stored in the Client's PC 30 and assigned to the audio call 
component and the data call component of client 30 (See Fig.1, segments 60 and 
61 of the screen shot in Surf & Call ® implementation and browser data 
component applet) and the audio call is initiated by client 30 routing the audio 
component to an agent 22 via Gatekeeper 38, PBX 36 and ACD 40. 

The agent called, say 22a, has a telephone extension number as its 
identifier for example 977 868, which is associated with the ANI identifier of the 
audio connection when Client 20 calls agent 22a and a connection is made 
through PBX 36 and ACD 40. 

When the audio call begins (step 60), the ACD (Automatic Call 
Distributor) 40 in the call center sends a CTI (Computer Telephony Integration) 
event to T-server (Telephony server) 42 (step 62). This event includes information 
on the co-surfer-identifiers of the agent and the customer. Both the ANI of the 
client and the telephone extension of the agent are in the startcall event. T-server 
42 sends the event to CTI driver 46 in DC server 44 via a TCP/IP link CTI driver 
46 passes it onto event handler 45 in DC server 44 

Event handler 45 receives the CTI event or "start call event " and creates 
and stores a named "conversation" (step 64) as defined hereinabove, between 
Client 20 and agent 22a. 

Code in DC server 44 that uses a hashtable then hashes each co-surfer 
identifier to a data structure representing the cosurfers connection to DC server 
44 (A cosurfer identifier is the ANI in the case of the Client 20 and the extension 
number (977 868) in the case of the Agent 22A). This is shown, by way of 
example, in Fig. 3A now referred to, which lists the conversations of all parties 
which have, in co-operation, created a start call event. Thus, for example, second 
client 21 might be connected via PBX 36 and ACD 40 to a different agent 22b and 
this would create a second "start call event " to create a second conversation to 
accompany the first conversation, as shown in Fig. 3A. 

The conversation is recorded regardless of whether a co-surfing 
connection or a data connection exists wherein, for the avoidance of doubt, a 
co-surfing connection is defined as both parties computers being connected to DC 
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server 44 and a data connection is defined as either party's computer being 
connected to DC server 44. 

The data component of the client 30 call is connected via the internet to 
Data Collaboration (DC) server 44 with the ANI attached, thus facilitating the 
required data to stream through DC server 44 with the ANI attached. The data 
component of the agent 22 is routed via the Internet 24 to DC server 44, utilizing 
the browser of the computer of agent 22 (step 66). Agent 22 manually sends his 
telephone number, 977 868, to the DC server 44 to identify his data component to 
DC server 44. "Manually" in this connotation may mean that agent 22 types in the 
telephone extension number to his browser, which is sent from Java Script 
through the DC connection to DC server 44. It could, however, mean that every 
time the browser is opened the telephone extension number Identification, 977 
868, is, for example, sent to DC server 44. This process is also described in the 
Collaboration Patent and the Call Center Patent. The identifiers for the data 
connections for the above conversations between, by way of example, Client 20 
and agent 22a on the one hand and between Client 21 and agent 22b on the 
other hand are recorded in a hashtable contained in DC server 44 which lists the 
data connections as shown in Fig. 3B, now referred to (step 68). 

In DC server 44 when , for example, Client 20 sends a data message to 
DC server 44software or hardware is utilized by DC server 44 to check that Client 
20 is in a conversation (step 70), DC server confirms Client 20 is in the 
conversation (see step 64). DC server 44 then identifies who else is in that 
conversation (see step 64) and identifies agent 22a as being in that conversation 
(step 72). DC server 44 then attempts to send data to agent 22a (step 74) and 
because agent 22a is connected and its identifier is in the connections hashtable 
(see step 66) data goes to agent 22a (step 76). Identification is achieved by DC 
server 44 by comparison of the respective identifiers of Client 20 and agent 22 
with identifiers in the conversation and connections hashtables. The part of DC 
server 44 which switches messages between all connected cosurfers in a 
conversation is message distributor 47. 

For example, Fig. 3A illustrates that the list of conversations includes a 
conversation, conversation 77 between Client 20 and agent 22a, with their 
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identifiers shown. Fig. 3B illustrates that the list of connections includes data 
connections, connection 78, between the browsers of Client 20 and agent 22a 
containing the identifiers of Client 20 and agent 22a. 

The comparative software analyzes that the identifier for the Client 20 is 
5 present in the conversation 77 when Client 20 sends a data message and that 
agent 22a is also in conversation 77. DC server 44 (via message distributor 47) 
then attempts to send the data to agent 22a and succeeds because agent 22a is 
connected, as shown in the connections table (connection 78). A data 
collaboration session thus occurs. Conversely, for client 21 and agent 22b, 

10 although Fig. 3A shows that they are in a conversation 77a, the browser of agent 
22b has been cut off, or is not yet connected and its identifier does not appear in 
the connections hashtable, of connection 78a. Data collaboration cannot, 
therefore, occur, even though DC server 44 attempts to send the data to agent 
22a However, so long as the audio link between client 21 and agent 22b is 

15 maintained, the conversation is maintained and is in the list of conversations with 
certain consequences. Simple message switching is used, utilizing message 
distributor 47, within DC server 44: when the DC server 44 receives a message 
from a cosurfer, it checks to see what conversation that cosurfer is in , if any and 
sends the message to all connected co-surfers in the conversation The audio call 

20 proceeds regardless of whether the Data Collaboration applets in the cosurfers 
(agent's and customer's) browser have connected to the data collaboration server 
and the order of connection of the applets in each of the co-surfer's browsers 
and/or the audio call in relation to each other is irrelevant. Thus, the audio call 
may occur before or after the browser of client 20 or the agent 22 has connected 

25 to DC server 44 , where the browser of the client 20 may connect before the 
browser of the agent 22 or visa versa. 

•By way of further emphasis, the existence of a conversation tells the 
system that these two parties are 'in conversation 1 or associated for co-surfing, 
even if no data actually passes in the DC component. 

30 Thus, whether the browsers are open, the computers are down or the 

browsers are not connected to DC server 44, if an audio conversation exists, 
cosurfing can occur between a client 20 and an agent 22a connected in an audio 
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conversation, immediately the required cosurfing or browser connections are 
made. 

Thus, the present invention can contend with a system in which the 
browser is closed and opened in mid-conversation. If a web-form has been 
partially filled out by client 20 (co surfer A) when the browser of agent 22a (co 
surfer B) is closed and then opened, the contents of the form from cosurfer A's 
browser are recopied into the agent's browser. This is done as follows: When the 
cosurfer B's browser is closed, DC server 44 notices that the connection between 
the cosurfer and DC server 44 is lost. Server 44 notifies the software in cosurfer 
A's browser that the other party in the conversation is not connected to DC server 
44. From this point on the software in cosurfer A's browser begins storing all 
data entered into the form (rather than attempting to send this data to cosurfer B). 
When cosurfer B next connects to DC server 44, then if the connection is still 
going, DC server 44 will notify the software in cosurfer A's browser that it is 
possible to converse with another party. The software in cosurfer A's browser will 
send the contents of the form to the software in cosurfer B's browser. 

Reference is now briefly made to Fig. 4 which is a schematic illustration of 
a second embodiment of the present invention. Items similar to those in previous 
figures have similar reference numerals and will not be described further. The 
present embodiment illustrates the use of a regular telephone, 80 with the present 
invention, dispensing with the ANI used in the first embodiment. Hence, when the 
audio call is made by Client 20 the audio component identifier relayed to the PBX 
36 is a regular telephone number (as is always the case with the identifier of 
agent 22a). The identifier of client 20 is then inputted "manually", as described 
hereinabove, in relation to the first embodiment by Client 20, into the browser of 
his computer and sent to DC server 44. It may be automatically downloaded to 
DC server 44 when his browser is opened. 

The methods and apparatus disclosed herein have been described with 
exemplary reference to specific hardware and/or software. The methods and 
apparatus have been described in a manner sufficient to enable persons of 
ordinary skill in the art to readily adapt other commercially available hardware and 
software as may be needed to reduce any of the embodiments of the present 
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invention to practice without undue experimentation and using conventional 
techniques. 

It will be appreciated, by persons skilled in the art, that the present 
invention is not limited by what has been particularly shown and described 
hereinabove. Rather the scope of the invention is defined by the claims that 
follow: 

) 
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CLAIMS 

1 . A method of communication between at least two entities having at least 
two types of communications system which an at least one communication 
system employing first type of communication by a first communication 
application and a second communication system employing a second type of 
communication by a second communication application, wherein the method 
comprising the steps of: 

initiating communications between the at least two entities characterized 

by 

establishing communications of a first type between the at 
least two first communication applications of the at least two of 
said entities; and 

sending a message for notifying of a start of 
communications of a first type between the at least two said 
entities to a server of said communication system of second type 
for enabling communication of said second type; 

and 

terminating communications between the at least two entities by steps 
including: 

sending a message for notifying of an end communications 
of said first type between at least two said entities to said server of 
said communication system of second type for disabling said 
second type of communication. 

2. A method as claimed in claim 1, wherein the step of sending a message 
for notifying of a start of communications of a first type between the at least two 
said entities comprises: 

at least one identification of the first entity of the at least two entities; and 
at least one identification of the second entity of the at least two entities. 
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3. A method as claimed in claim 2, wherein the step of sending a message 
for notifying of an end of communications of said first type between at least two 
said entities comprises: 

at least one identification of the first entity of the at least two entities; and 
at least one identification of the second entity of the at least two entities. 

4. A method as claimed in claims 1 additionally comprising the step of: 
establishing said communications between the at least two entities 

characterized by 

establishing a connection between the first entity second 
communication application to the server of the communication system of 
the second type; 

establishing a connection between the second entity second 
communication application to the server of the communication system of 
the second type; and 

transferring communications between at least two entities second 
communication application by attaching the at least one identification of 
the second entity to the first entity communications and the at least one 
identification of the first entity to the second entity communications by 
said server of the communication system of the second type. 

5. A method as claimed in claim 1 , wherein communications of the first type 
include voice communications. 

6. A method as claimed in claim 1 , wherein communications of the second 
type include a data communications. 

7. A method as claimed in claim 1, wherein said server of the second 
communication system is a data collaboration server. 

8. A method as claimed in claims 1, wherein communications between the 
at least two entities comprises at least voice communications and data 
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communications and the initiating and the terminating of said data 
communications are in accordance with the initiating and terminating of said 
voice communications. 

9. A method as claimed in each one of the claim 1 , wherein the first 
communication system further comprises a server for sending messages for 
notifying the start and end of communications of the first type to the server of 
the communication system of the second type. 

10. A method as claimed in each one of the claim 1 wherein the server of the 
first communication system is a telephony server. 

11. A communication system for providing communications between an at 
least two entities, which the communications include an at least two types of 
communications and wherein the system comprising: 

first means for providing a first type of communications; 

second means for providing a second type of communications 

an entity from the at least two entities for initiating said communications, 

wherein said first means is configured for 

providing a first communication application for establishing 
communications of a first type; and 

providing a communication server for sending a message of 
a start of a first type of communications between at least two of 
said entities to a server of said second type of communication 
system for enabling said second type of communication and for 
sending a message of an end of said first type of communications 
to said server of said second communication system for disabling 
said second type of communication. 

12. A communication system as claimed in claim 11, additionally comprising: 
means for establishing said communications, said communication 

establishing means including: 
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a first communication device for providing communications of a the 
second type for establishing a connection between the first entity to" the 
server of the communication system of the second type; 

a second communication device application for providing 
communications of a the second type for establishing a connection 
between the second entity to the server of the communication system of 
the second type; and 

means for transferring communications of said second type 
between at least two entities, said transferring means configured for 
attaching the at least one identification of the second entity to the first 
entity communications and the at least one identification of the first entity 
to the second entity communications. 

13. A communication system as claimed in claim 11, adapted for 
communications of the first type are a voice communications. 

14. A communication system as claimed in claim 1 1 , adapted for 
communications of the second type are a data communications. 

15. A communication system as claimed in claim 11, wherein said server of 
the communication of second type includes a data collaboration server. 

16. A communication system as claimed in claim 1 1 , wherein said server of 
the communication system of the first type includes a telephony server. 
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